Combining a Meta-Policy and Monte-Carlo Planning for Scalable Type-Based Reasoning in Partially Observable Environments

The design of autonomous agents that can interact effectively with other agents without prior coordination is a core problem in multi-agent systems. Type-based reasoning methods achieve this by maintaining a belief over a set of potential behaviours for the other agents. However, current methods are limited in that they assume full observability of the state and actions of the other agent or do not scale efficiently to larger problems with longer planning horizons. Addressing these limitations, we propose Partially Observable Type-based Meta Monte-Carlo Planning (POTMMCP) - an online Monte-Carlo Tree Search based planning method for type-based reasoning in large partially observable environments. POTMMCP incorporates a novel meta-policy for guiding search and evaluating beliefs, allowing it to search more effectively to longer horizons using less planning time. We show that our method converges to the optimal solution in the limit and empirically demonstrate that it effectively adapts online to diverse sets of other agents across a range of environments. Comparisons with the state-of-the art method on problems with up to $10^{14}$ states and $10^8$ observations indicate that POTMMCP is able to compute better solutions significantly faster.Comment: 24 page

Workspace-based sampling for probabilistic path planning

Ph.DDOCTOR OF PHILOSOPH

Global Motion Planning under Uncertain Motion, Sensing, and Environment Map

Motion planning that takes into account uncertainty in motion, sensing, and environment map, is critical for autonomous robots to operate reliably in our living spaces. Partially Observable Markov Decision Processes (POMDPs) is a principled and general framework for planning under uncertainty. Although recent development of point-based POMDPs have drastically increased the speed of POMDP planning, even the best POMDP planner today, fails to generate reasonable motion strategies when the environment map is not known exactly. This paper presents Guided Cluster Sampling (GCS), a new point-based POMDP planner for motion planning under uncertain motion, sensing, and environment map, when the robot has active sensing capability. It uses our observations that in this problem, the belief space B can be partitioned into a collection of much smaller subspaces, and an optimal policy can often be generated by sufficient sampling of a small subset of the collection. GCS samples B using two-stage cluster sampling, a subspace is sampled from the collection and then a belief is sampled from the subspace. It uses information from the set of sampled sub-spaces and sampled beliefs to guide subsequent sampling. Preliminary results suggest that GCS generates reasonable policies for motion planning problems with uncertain motion, sensing, and environment map, that are unsolvable by the best point-based POMDP planner today, within reasonable time. Furthermore, GCS handles POMDPs with continuous state, action, and observation spaces. We show that for a class of POMDPs that often occur in robot motion planning, GCS converges to the optimal policy, given enough time. To the best of our knowledge, this is the first convergence result for point-based POMDPs with continuous action space

The Role of Hyperlactatemia Status as a Prognostic Parameter in Critically Ill Nenonates

Introduction: Hypoxia and hypoperfussion is often found in neonates in an intensive care unit, however the clinical manifestations can only be found after cellular hypoxia and tissue perfussion disorder occur. Objective: The study aims to find the ability of hyperlactatemia status as a prognostic parameter for infants in Neonatal Intensive Care Unit Level IIIA. Methods : The research was a cohort prospective study using bivariat and multivariat analysis in NICU of Dr. Wahidin Sudirohusodo Hospital from June 2012 to April 2013. The analysis of the lactate level on samples that met the inclusion and exclusion criterias was done immediately after the neonates get into the NICU level IIIA. The capliary lactate level was measured using hand held analyser. The samples were distributed into groups of hyperlactatemia and without hyperlactatemia followed by outcome observation (death or good recovery). The number of subjects were 102 patients consisted of 69 males and 33 females. Results : The study showed the incidence of hyperlactatemia at NICU leve IIIA was 53,9%. Hyperlactatemia ((p=0.000; IK95% 4.11-56.75.00; AOR 15.28) and chronological age <24 hours (p=0.014; IK95% 1.5037.04; AOR 7.47) was significant in determining the patient\u27s outcome. Conclusions: The study found that hyperlactatemia status and cronological age less than 24 hours were prognostic factors for patient\u27s outcome related to elevated mortality risk

Experiments on Surface Reconstruction for Partially Submerged Marine Structures

Over the past 10 years, significant scientific effort has been dedicated to the problem of three-dimensional (3-D) surface reconstruction for structural systems. However, the critical area of marine structures remains insufficiently studied. The research presented here focuses on the problem of 3-D surface reconstruction in the marine environment. This paper summarizes our hardware, software, and experimental contributions on surface reconstruction over the past few years (2008–2011). We propose the use of off-the-shelf sensors and a robotic platform to scan marine structures both above and below the waterline, and we develop a method and software system that uses the Ball Pivoting Algorithm (BPA) and the Poisson reconstruction algorithm to reconstruct 3-D surface models of marine structures from the scanned data. We have tested our hardware and software systems extensively in Singapore waters, including operating in rough waters, where water currents are around 1–2 m/s. We present results on construction of various 3-D models of marine structures, including slowly moving structures such as floating platforms, moving boats, and stationary jetties. Furthermore, the proposed surface reconstruction algorithm makes no use of any navigation sensor such as GPS, a Doppler velocity log, or an inertial navigation system.Singapore-MIT Alliance for Research and Technology. Center for Environmental Sensing and Modelin

Infrastructure for 3D model reconstruction of marine structures

3D model reconstruction of marine structures, such as dams, oil-rigs, and sea caves, is both important and challenging. An important application includes structural inspection. Manual inspection of marine structures is tedious and even a small oversight can have severe consequences for the structure and the people around it. A robotic system that can construct 3D models of marine structures would hopefully reduce the chances of oversight, and hence improve the safety of marine environment. Due to the water currents and wakes, developing a robotic system to construct 3D models of marine structures is a challenge, as it is difficult for a robot to reach the desired scan configurations and take a scan of the environment while remaining stationary. This paper presents our preliminary work in developing a robotic and software system for construction of 3D models of marine structures. We have successfully tested our system in a sea water environment in the Singapore Straits